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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH (S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

i - If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 12 July 2004 . 
2a)D This action is FINAL. 2b)[x] This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 1^ Claim(s) 1-38 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-38 is/are rejected. 

7) D Claim(s) „ is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) Q Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)D All b)D Some * c)\J None of: 

1 □ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) □ Notice of ,nformal Patent Application (PTO-1 52) 
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DETAILED ACTION 

1 . The Request for Continued Examination (RCE) filed on 12 July 2004 under 37 CFR 
1 .53(d) based on parent Application No. 09/866,394 is acceptable and a RCE has been 
established. An action on the RCE follows. 

2. The amendments submitted with the filing of the RCE on 12 July 2004 have been 
received and entered. Claims 1-38 as amended are pending in the application. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 1-38 are rejected under 35 U.S.C. 103(a) as being unpatentable over the article 
entitled "Color SuperHistograms for Video Representation", written by Dimitrova et al., and 
Wang et al. U.S. Patent 5,805,733. 

Referring to claims 1, 11,21 and 30, Dimitrova et al. teach an apparatus, system, method 
and computer executable instructions comprising a visual summary controller capable of creating 
a visual summary of video material (Dimitrova et al.: page 316, Figure 1), wherein the visual 
summary controller is capable of extracting frame signatures (histograms) from keyframes of 
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video material and capable of using the frame signatures to create superhistograms from the 
keyframes (Dimitrova et al.: page 314, right column, lines 1 1-25, page 315, section 2 and page 
316, section 2.3; this is further shown in Figure 1). However, although Dimitrova et al. teach 
using the frame signatures and superhistograms to create a visual summary of video material in a 
broad sense (representing video segments by computing superhistograms) (Dimitrova et al.: 
Abstract), Dimitrova et al. fail to explicitly teach selecting certain keyframe images to create a 
compact visual summary of the video material Wang et al teach the analysis of scenes and 
frames in video materials (Wang et al.: column 1, lines 53-56 and Figure 2) similar to that of 
Dimitrova et al. In addition, Wang et al. further teach selecting certain keyframe images to 
create a compact visual summary of the video material (summarizing a video sequence by taking 
one representative frame from each set of related scenes with similar average color histograms, 
to represent the set to enable the user to view a large sampling of video sequence images) (Wang 
et al: column 1, lines 51-67 and column 2, lines 1-24; this is further shown in Figure 3). It 
would have been obvious to one of ordinary skill in the art, having the teachings of Dimitrova et 
al and Wang et al. before him at the time the invention was made, to modify the visual summary 
controller capable of extracting frame signatures from keyframes to create superhistograms 
taught by Dimitrova et al, to include the further step of using certain keyframe images to create 
a compact visual summary of Wang et al. One would have been motivated to make such a 
combination in order to meet the need of being able to readily access and manipulate video 
information, by cataloguing and storing the potentially thousands of hours of video for rapid 
future retrieval, browsing and use, created by the increasing availability and use of digital video 
and the increasing integration of computer technologies and video production technologies. 
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Referring to claims 2, 12, 22 and 31, Dimitrova et al. teach the filtering of keyframes 
(merging of histograms into family histograms) and extracting frames signatures (computing 
color histograms) from the filtered keyframes before using the frame signatures (histograms) to 
create the superhistogram representing a visual summary of the video material (page 315, right 
column, section 2 and page 316, left column, section 2.3). 

Referring to claims 3, 13, 23 and 32, Dimitrova et al. teach the use of superhistograms to 
cluster the filtered keyframes (the ordered merging of the family histograms to create the 
superhistogram), wherein the clustered keyframes (superhistogram) represents the visual 
summary of the video material, as recited on page 314, right column, lines 1 1-25 and shown in 
Figure 1. 

Referring to claims 4 and 14, Dimitrova et al. teach the use of a histogram as the frame 
signature used to compute superhistograms (page 314, right column, lines 11-15). . 

Referring to claims 5, 15, 24 and 33, Dimitrova et al. the use of the LI distance measure 
method, L2 distance measure method, histogram intersection method, Chi-Square test and Bin- 
wise histogram intersection method to computer the histogram difference (page 3 1 5, right 
column). 

Referring to claims 6, 16, 25 and 34, Dimitrova et al. teach the selection of the most 
meaningful image for each superhistogram (the top n largest families) as the representative 
image (page 316, section 2.4). 

Referring to claims 7, 17, 26 and 35, Dimitrova et al teach the ability to select the family 
histograms (the top n largest families) to use to create the superhistogram used to create the 
visual summary (page 316, section 2.4). 
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Referring to claims 8, 18, 27 and 36, while Dimitrova et al. teach all of the limitations as 
applied to claims 1, 11,21 and 30 above, Dimitrova et al. fail to explicitly teach the capability to 
retrieve a visual summary stored in a memory unit and causing the visual summary to be 
displayed in response to a user request. Wang et al teach the analysis of scenes and frames in 
video materials (Wang et al: column 1, lines 53-56 and Figure 2) similar to that of Dimitrova et 
al. In addition, Wang et al. further teach the capability of letting a user select a visual summary 
for viewing, retrieving that visual summary from memory and displaying it in response to the 
user's request (displaying visual summaries of scenes in a movie bar and allowing users to 
access the summaries by selecting the segments corresponding to the scenes) (Wang et al: 
column 2, lines 16-29 and shown in Figures 2 and 3). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Dimitrova et al. and Wang et al. before him at 
the time the invention was made, to modify the visual summary controller capable of extracting 
frame signatures from keyframes to create superhistograms of Dimitrova et al., to include the 
retrieval and display of the visual summary in response to a user request, as taught by Wang et 
al. One would have been motivated to make such a combination to give users the flexibility to 

select which scenes to watch, saving them time from having to browse through all of the other 

j 

irrelevant scenes; furthermore, because the increasing availability and use of digital video and 
the increasing integration of computer technologies and video production technologies have 
produced the need to be able to readily access and manipulate video information, it would have 
been advantageous to make such a combination in order to provide users a way to summarize the 
content of video quickly and easily, in order to catalogue and store the potentially thousands of 
hours of video for rapid future retrieval, browsing and use. 
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Referring to claims 9, 19, 28 and 37, Dimitrova et al. teach the use of the visual summary 
obtained from the superhistograms to access at least a portion of the video material (classifying 
and searching in video archives), as recited on page 317, section 4.2. 

Referring to claim 10, 20, 29 and 38, while Dimitrova et al teach all of the limitations as 
applied to claims 1, 1 1, 21 and 30 above, Dimitrova et al. fail to explicitly teach the creation of 
new video material using the compact visual summaries. Wang et al. teach the analysis of scenes 
and frames in video materials (Wang et al: column 1, lines 53-56 and Figure 2) similar to that of 
Dimitrova et al. In addition, Wang et al. further teach the creation of new video material using 
the compact visual summaries (a collage made up of representative frames for each set of 
summarized scenes) (Wang et al.: column 3, lines 53-57). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Dimitrova et al. and Wang et al before him at 
the time the invention was made, to modify the visual summary controller capable of extracting 
frame signatures from keyframes to create superhistograms of Dimitrova et al., to include the 
creation of new video material, as taught by Wang et al. It would have been advantageous for 
one to utilize such a combination in order to conserve processor time and storage space by 
utilizing the already existing visual summaries in the creation of new visual materials; 
furthermore, because the increasing availability and use of digital video and the increasing 
integration of computer technologies and video production technologies have produced the need 
to be able to readily access and manipulate video information, it would have been advantageous 
to make such a combination in order to provide users a way to summarize the content of video 
quickly and easily, in order to catalogue and store the potentially thousands of hours of video for 
rapid future retrieval, browsing and use. 
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4. The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. §1.11 1(c) to consider 
these references fully when responding to this action. The documents cited therein teach similar 
methods of using keyframes to create visual summaries of video material. 

Response to Arguments 

5. Applicants arguments with respect to claims 1^38 have been considered but are moot in 
view of the new ground(s) of rejection. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (703) 305-0328 through 
the month of October, 2004 and (571) 272-4058 thereafter. The examiner can normally be 
reached on Monday - Friday, 7:30 am - 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached at (703) 308-3 116 through the month of October, 2004 
and at (571) 272-4048 thereafter. The fax phone number for the organization where this 
application or proceeding is assigned is (703) 746-8720 through the month of October, 2004 and 
(571) 273-4058. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact / t&e^lectronic Business Center (EBC) at 866-2 1 7-9 1 97 (toll-free). 




